A Closer Look at Debiased Temporal Sentence Grounding in Videos: Dataset, Metric, and Approach

Temporal Sentence Grounding in Videos (TSGV) , which aims to ground a natural language sentence that indicates complex human activities an untrimmed video, has drawn widespread attention over the past few years. However, recent studies have found current benchmark datasets may obvious moment annotation biases, enabling several simple baselines even without training achieve state-of-the-art (SOTA) performance. In this paper, we take closer look at existing evaluation protocols for TSGV, and find both prevailing dataset splits metrics are devils lead untrustworthy benchmarking. Therefore, propose re-organize two widely-used datasets, making ground-truth distributions different test splits, i.e., out-of-distribution (OOD) test. Meanwhile, introduce new metric “dR@ n ,IoU= m ” discounts basic recall scores especially with small IoU thresholds, so as alleviate inflating caused by biased large proportion of long moments. New benchmarking results indicate our proposed can better monitor research progress TSGV. Furthermore, novel causality-based Multi-branch Deconfounding Debiasing (MDD) framework unbiased prediction. Specifically, design multi-branch deconfounder eliminate effects multiple confounders causal intervention. order help model align semantics between queries video moments, enhance representations during feature encoding. textual information, query is parsed into verb-centered phrases obtain more fine-grained feature. For visual positional information been decomposed from features moments diverse locations. Extensive experiments demonstrate approach competitive among SOTA approaches outperform base great gains.